Assistive Device for Non-Visually Discerning a Three-Dimensional (3D) Real-World Area Surrounding a User

ABSTRACT

An assistive device and method for non-visually discerning a three-dimensional (3D) real-world area surrounding a user, comprises a haptic feedback interface that includes a plurality of haptic elements. The assistive device receives sensor data of the 3D real-world area within a first proximity range of the assistive device from a plurality of different types of sensors that are communicatively coupled to the assistive device. The assistive device establishes a mapping of a plurality of objects within the first proximity range to the plurality of haptic elements in a defined region of the haptic feedback interface, based on the received sensor data. A haptic feedback generator generates a touch-discernible feedback on the haptic feedback interface based on the established mapping. The touch-discernible feedback comprises a plurality of differential touch-discernible cues to enable the user to non-visually discern the 3D real-world area surrounding the user.

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE

None.

FIELD

Various embodiments of the disclosure relate to assistive technologies.More specifically, various embodiments of the disclosure relate to anassistive device and method for non-visually discerning athree-dimensional (3D) real-world area surrounding a user.

BACKGROUND

With the growth of human-machine interaction (HMI) and sensortechnologies, various types of assistive devices have been developed.However, technological developments in HMI are mostly focused onvision-based interaction technology. Humans have five traditionalrecognized senses, sight (ophthalmoception), hearing (audioception),taste (gustaoception), smell (olfacoception or olfacception), and touch(tactioception). The loss of one or more senses generally results inenhancement of one or more of the remaining senses to compensate for thelost sense(s). For people that have loss or impaired sight, existingtechnology are typically focused on Braille based tactile presentationsystems. A Braille character includes a fixed pattern, which is athree-row by two-column “6-dot Braille cell” or a four-row by two-column“8-dot Braille cell” with combinations of raised dots that representsletters of the alphabet, numbers and punctuation, and defined symbols.As existing technology are typically focused on Braille based tactilepresentations, HMI for people that have loss or impaired sight areusually limited to use of 8-keys Braille input, on-screen readers, orother tactile forms that are of limited functionality and use. It isknown that the sense of touch has a much greater sensory resolution thanthe sense of sight. Hence, the sense of touch can detect even smallchanges on a surface that the eye cannot detect. These powerfulnon-visual senses, such as the sense of touch or hearing, maypotentially be harnessed to help people that have lost or impaired thesense of sight to better understand and navigate in the world in aneffective manner. These powerful non-visual senses may also be used incertain situations where human vision is of limited use, for example, inareas that are devoid or partially devoid of light. Thus, an advancedsystem may be required for non-visually discerning a three-dimensional(3D) real-world area surrounding a user.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of skill in the art, throughcomparison of described systems with some aspects of the presentdisclosure, as set forth in the remainder of the present application andwith reference to the drawings.

SUMMARY

An assistive device and method for non-visually discerning athree-dimensional (3D) real-world area surrounding a user substantiallyas shown in, and/or described in connection with, at least one of thefigures, as set forth more completely in the claims.

These and other features and advantages of the present disclosure may beappreciated from a review of the following detailed description of thepresent disclosure, along with the accompanying figures in which likereference numerals refer to like parts throughout.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary environment for non-visually discerninga three-dimensional (3D) real-world area surrounding a user, inaccordance with an embodiment of the disclosure.

FIG. 2 is a block diagram that illustrates an exemplary assistive devicefor non-visually discerning a 3D real-world area surrounding a user ofthe assistive device, in accordance with an embodiment of thedisclosure.

FIGS. 3A, 3B, 3C, and 3D illustrate exemplary scenario diagrams forimplementation of the assistive device and method for non-visuallydiscerning a 3D real-world area surrounding a user, in accordance withan embodiment of the disclosure.

FIGS. 4A and 4B, collectively, depict a flow chart that illustrates amethod for non-visually discerning a 3D real-world area surrounding auser, in accordance with an embodiment of the disclosure.

DETAILED DESCRIPTION

The following described implementations may be found in the disclosedassistive device and method for non-visually discerning a 3D real-worldarea surrounding a user. Exemplary aspects of the disclosure may includean assistive device that may include a haptic feedback interfacecomprising a plurality of haptic elements. The assistive device mayfurther comprise a first circuitry configured to receive sensor data ofa 3D real-world area within a first proximity range of the assistivedevice from a plurality of different types of sensors that arecommunicatively coupled to the assistive device. The assistive devicemay further comprise a second circuitry configured to establish amapping of a plurality of objects within the first proximity range tothe plurality of haptic elements in a defined region of the hapticfeedback interface, based on the received sensor data. The assistivedevice may further include a haptic feedback generator configured togenerate a touch-discernible feedback on the haptic feedback interfacebased on the established mapping. The touch-discernible feedback maycomprise a plurality of differential touch-discernible cues to discern a3D arrangement of each of the plurality of objects with respect to aposition of a user of the assistive device. The plurality ofdifferential touch-discernible cues refers to touch-discernible cuesgenerated on the haptic feedback interface that may be dissimilar ordifferent in one or more physical characteristics or properties suchthat a user may discern their disparate form by tacioperception. The oneor more physical characteristics or properties, for example, may beshape, size, smoothness, roughness, temperature, amount of pressure,pain, level of elevation of a protrusion, a pattern of distribution ofthe cues, and the like, which may be discernible by touch.

In accordance with an embodiment, the touch-discernible feedback maycorrespond to at least one of a differential pressure-based, adifferential temperature-based, a differential electric pulse-based, adifferential raised shape pattern-based touch-discernible feedback, or acombination of different touch-discernible feedbacks. The secondcircuitry may be further configured to identify an object-type of eachof the plurality of objects present within the first proximity range ofthe assistive device based on the received sensor data. The hapticfeedback generator may be further configured to generate the pluralityof differential touch-discernible cues to discern different identifiedobject-types of the plurality of objects present within the firstproximity range of the assistive device.

In accordance with an embodiment, the second circuitry may be furtherconfigured to determine a relative position and a height of each of theplurality of objects with respect to the position of the user of theassistive device for the 3D arrangement. The relative position of eachof the plurality of objects with respect to the position of the user ofthe assistive device may be determined based on the sensor data receivedin real time or near-real time from the plurality of different types ofsensors. In some embodiments, the plurality of different types ofsensors may be worn by the user.

In accordance with an embodiment, the second circuitry may be furtherconfigured to generate a 3D digital model of the 3D real-world areawithin the first proximity range, based on the received sensor data. Thegenerated 3D digital model may be utilized for the mapping of theplurality of objects within the first proximity range to the pluralityof haptic elements. In some embodiments, the second circuitry may beconfigured to acquire a first template map of the 3D real-world areawithin the first proximity range of the assistive device from a serverbased on the position of the user. The first template map may then beupdated with at least positional information of the plurality ofobjects. The update may be done based on the sensor data of the 3Dreal-world area within the first proximity range of the assistive devicereceived from the plurality of different types of sensors in real timeor near-real time.

In accordance with an embodiment, the circuitry may be furtherconfigured to determine a scaling factor in accordance with the firstproximity range selected on the assistive device for the mapping of theplurality of objects within the first proximity range to the pluralityof haptic elements in the defined region of the haptic feedbackinterface. The mapping of the plurality of objects within the firstproximity range may be updated to the plurality of haptic elements inthe defined region of the haptic feedback interface. The mapping updatemay be done based on a change in position of one or more movable objectsof the plurality of objects or the user, detected in real time ornear-real time from the received sensor data.

In accordance with an embodiment, the haptic feedback generator of theassistive device may be configured to dynamically update thetouch-discernible feedback on the haptic feedback interface based on theupdate of the mapping. The updated touch-discernible feedback maycomprise a change in the relative position of one or more of theplurality of objects with respect to the position of the user of theassistive device. In some embodiments, the haptic feedback generator maybe further configured to output an audio feedback in combination withthe touch-discernible feedback to enable non-visual discern of the 3Dreal-world area within the first proximity range of the assistive deviceby the user as the user moves from a first location to a second locationin the 3D real-world area within the first proximity range.

In accordance with an embodiment, the assistive device may comprise alearning engine in a memory of the assistive device. The secondcircuitry may be further configured to determine one or more patterns ina plurality of user interactions on the haptic feedback interface over aperiod of time based on monitoring or tracking of a usage pattern of theassistive device by the learning engine. The second circuitry may befurther configured to adapt the mapping of the plurality of objectswithin the first proximity range to the plurality of haptic elements inthe defined region of the haptic feedback interface based on thedetermined one or more patterns. In accordance with an embodiment, thehaptic feedback interface is a haptic input/output interface. In someembodiments, the assistive device further comprises a haptic feedbackcontroller. The haptic feedback controller may be configured to detect ahaptic input on the haptic feedback interface, based on a press on atleast one of the plurality of differential touch-discernible cuesgenerated in the touch-discernible haptic feedback.

FIG. 1 illustrates an exemplary environment for non-visually discerninga three-dimensional (3D) real-world area surrounding a user, inaccordance with an embodiment of the disclosure. With reference to FIG.1, there is shown an exemplary environment 100. The exemplaryenvironment 100 may include an assistive device 102, a plurality ofsensors 104, a server 106, a first communication network 108A, a secondcommunication network 108B, and one or more users, such as a user 110.The assistive device 102 may include a haptic feedback interface 112.The assistive device 102 may be communicatively coupled to the pluralityof sensors 104 via the first communication network 108A or the secondcommunication network 108B. The assistive device 102 may becommunicatively coupled to the server 106 via the second communicationnetwork 108B.

The assistive device 102 may refer to a special-purpose device toprovide assistance to a user, such as the user 110, of the assistivedevice 102 for non-visually discerning any 3D real-world areasurrounding the user 110. The 3D real-world area surrounding the user110 may be an indoor area or an outdoor area. The assistive device 102may include suitable logic, circuitry, and/or code to generate atouch-discernible feedback on the haptic feedback interface 112. Thetouch-discernible feedback on the haptic feedback interface 112 mayenable the user 110 to discern the 3D real-world area surrounding theuser 110. In accordance with an embodiment, the touch-discerniblefeedback may be 3D touch-discernible feedback generated based on thesensor data of the 3D real-world area surrounding the user 110 receivedfrom the plurality of sensors 104.

In some embodiments, the assistive device 102 may be implemented as ahand-held portable device. In some embodiments, the assistive device 102may be implemented as one or more wearable devices that may be wornaround at different parts of the human body having sensory receptorsthat senses touch. It is known that the somatic sensory system of humanbody is responsible for the sense of touch. The somatic sensory systemhas sensory touch or pressure receptors that enable a human to feel whensomething comes into contact with skin. The sense of touch may also bereferred to as somatic senses or somesthetic senses that includeproprioception (e.g. sense of position and movement) or hapticperception. Typically, such sensory receptors for sense of touch arepresent, for example, on the skin, epithelial tissues, muscles, bonesand joints, and even on certain internal organs of the human body.Examples of implementation of the assistive device 102 may include, butare not limited to a special-purpose portable assistive device,special-purpose hand gloves, special-purpose shoes, or a wearable devicethat may be worn as a wrist band, wrapped around arms, or any part ofhuman body or as a shoe sole.

The plurality of sensors 104 may comprise suitable logic, circuitry,and/or interfaces that may be configured to detect one or more cues ofthe 3D real-world area surrounding the user 110, and generate acorresponding output, such as sensor data. The plurality of sensors 104may include wearable sensors that may be worn by the user 110, sensorsthat may be integrated with the assistive device 102, or other personaldevices, such as a smartphone, of the user 110. The plurality of sensors104 refers to a plurality of different types of sensors. Examples of theplurality of sensors 104 may include, but are not limited to, a motionsensor (such as an accelerometer and a gyroscope), a location sensor(such as a global positioning system (GPS) sensor), a directiondetecting sensor (such as a compass or magnetometer), an image-capturedevice (such as a stereoscopic camera, 360 degree camera, a wide-anglecamera, or other image sensors), an atmospheric pressure detectionsensor (such as a barometer), a depth sensor, an altitude detectionsensor (such as altimeter), a lux meter, a radio frequency (RF) sensor,an ultrasound sensor, or an object detection sensor (such as Radar,Light Detection and Ranging (LIDAR),and an infrared (IR) sensor).

The server 106 may comprise suitable logic, circuitry, interfaces,and/or code that may be configured to store satellite imagery, streetmaps, and 360 degree panoramic views of streets of various geographicalareas. In some embodiments, the server 106 may be configured tocommunicate a first template map of the 3D real-world area for alocation of the assistive device 102, based on a template map requestfor the location received from the assistive device 102. In accordancewith an embodiment, the server 106 may be configured to store historicalusage pattern data of a plurality of different users, such as the user110. Examples of the server 106 may include, but are not limited to, acloud server, an application server, a database server, a web server, afile server, and/or their combination.

The first communication network 108A may be a medium that may enablecommunication between the assistive device 102 and the plurality ofsensors 104. The first communication network 108A may be implemented byone or more wired or wireless communication technologies known in theart. The first communication network 108A may refer to a short-range ormedium-range wireless communication network. Examples of wirelesscommunication networks may include, but are not be limited to, aWireless-Fidelity (Wi-Fi) based network, a Light-Fidelity (Li-Fi) basednetwork, a wireless personal area network (WPAN) such as a Bluetoothnetwork, Internet-of-Things (IoT) network, Machine-Type-Communication(MTC) network, and/or a Wi-Max based network.

The second communication network 108B may be a medium that mayfacilitate communication between the assistive device 102 and the server106. The second communication network 108B may be implemented by one ormore wireless communication technologies known in the art. Examples ofthe wireless communication networks may include, but not limited to, theInternet, a cloud network, a wireless wide area network (WWAN), a LocalArea Network (LAN), a plain old telephone service (POTS), a MetropolitanArea Network (MAN), or a cellular or mobile network, such as GlobalSystem for Mobile Communications (GSM), General Packet Radio Service(GPRS), Enhanced Data Rates for GSM Evolution (EDGE), 1G, 2G, 3G, 4GLong Term Evolution (LTE), 5G, IEEE 802.11, 802.16, and the like.

The haptic feedback interface 112 may comprise a plurality of hapticelements. In accordance with an embodiment, the haptic feedbackinterface 112 may refer to a haptic output interface configured toprovide at least a touch-discernible output to the user 110. In someembodiments, the haptic feedback interface 112 may refer to a hapticinput/output (I/O) interface configured to receive haptic input as wellas provide haptic output to the user 110 from the same haptic I/Ointerface. It is known that the sense of touch has a much greatersensory resolution than the sense of sight. Hence, the sense of touchcan detect even small changes on a surface that the eye cannot detect.This principle of the sense of touch may be used to guide the design ofthe haptic feedback interface 112.

In accordance with an embodiment, the user 110 may be a person who havelost or impaired the sense of sight. The user 110 may want to learn andunderstand about the surrounding world. It is known that sighted peoplevisualize the surrounding world by detection of edges between areas ofdifferent wavelengths of light, which is then perceived as differentcolors by brain. Based on feedback from the visual system, visual partof the brain referred to as visual cortex, processes visual informationof the surrounding world to enable the sighted people to visualize thesurrounding world. Information about the features on the surface of anobject, like color and shading, provide certain clues about itsidentity. It is also known the loss of one or more senses, such as thesense of sight, generally results in enhancement of one or more of theremaining senses, such as sense of touch, hearing, smell, or taste, tocompensate for the lost sense(s). The assistive device 102 harnesses thenon-visual senses, such as the sense of touch, hearing, or smell, toassist users, such as the user 110, who have lost or impaired the senseof sight for enhanced and accurate understanding of the 3D real-worldarea surrounding the user 110. The assistive device 102 may also be usedeven by sighted people in certain situations where human vision is oflimited use, for example, in areas that are devoid or partially devoidof light, for example, during night to augment sense of sight usingother human senses, such as audioception, olfacoception, andtactioception.

In operation, the assistive device 102 may be configured to receivesensor data of the 3D real-world area within the first proximity rangeof the assistive device 102 from a plurality of different types ofsensors (such as the plurality of sensors 104) that are communicativelycoupled to the assistive device 102. The plurality of different types ofsensors, for example, may include the location sensor, the motionsensor, the image-capture device, the RF sensor, the ultrasound sensor,the IR sensor, or other types of object detection sensor (such as Radaror LIDAR). The image-capture device may refer to a stereoscopic camera,360-degree camera, a night vision camera, a wide-angle camera, or otherimage sensors or their combination. Thus, in certain scenarios, whereone type of sensor may not capture accurate information of the 3Dreal-world area within the first proximity range of the assistive device102, other types of sensors may compliment and capture information ofthe 3D real-world area.

In accordance with an embodiment, the plurality of different types ofsensors (such as the plurality of sensors 104) may include sensors, forexample, rain sensors, altimeter, lux meter, barometer, and the like,that senses environmental conditions and/or characteristics, such asweather conditions or lighting conditions). Based on the environmentalconditions and/or characteristics, information of the 3D real-world areaacquired from a first group of sensors of the plurality of differenttypes of sensors may be assigned a higher weigh value (i.e. preferable)than information acquired from a second group of sensors of theplurality of different types of sensors. The classification of sensorsin the first group of sensors and the second group of sensors may bedone based on defined criteria and the sensed environmental conditionsand/or characteristics. The defined criteria, for example, may bedefined rules based on known accuracy of information detected indifferent environment conditions from each sensor. For example, incertain weather condition, the information, such as images captured fromthe image-capture device may not be useful. In such cases, the sensordata from the RF sensor, LIDAR, ultrasound sensor, may be providedhigher weight value as compared to the sensor data from theimage-capture device.

The assistive device 102 may be configured to identify the object-typeof each of the plurality of different objects present within the firstproximity range of the assistive device 102 based on the received sensordata. The assistive device 102 may be configured to determine a relativeposition of each of the plurality of objects with respect to theposition of the user 110 of the assistive device 102. The relativeposition of each of the plurality of objects may be determined based onthe sensor data received in real time or near-real time from theplurality of sensors 104. In some embodiments, the assistive device 102may be configured to determine a height of each of the plurality ofobjects from the perspective of the user 110 of the assistive device102. As the sensor data is received from different input sources (i.e.the plurality of different types of sensors), the information from eachsensor may be processed concurrently and information from one sensor maycompliment information from other sensor, thereby increasing accuracy ofidentification of the object-type, and the relative position of each ofthe plurality of objects with respect to the position of the user 110 ofthe assistive device 102.

In accordance with an embodiment, the assistive device 102 may beconfigured to compute a scaling factor in accordance with the firstproximity range and an area of the haptic feedback interface 112 onwhich a haptic feedback is to be generated. The assistive device 102 maybe configured to generate a 3D digital model of the 3D real-world areawithin the first proximity range. The 3D digital model of the 3Dreal-world area surrounding the user 110 may be generated based on thereceived sensor data. The determined relative position, the height, andthe identified object-type of each of the plurality of differentobjects, may also be used to generate the 3D digital model of the 3Dreal-world area surrounding the user 110. As the sensor data is receivedfrom different input sources (i.e. the plurality of different types ofsensors), the computation of the relative position of each of theplurality of objects with respect to the position of the user 110 of theassistive device 102, may be faster and more accurate as compared tosensor data received exclusively from one type of sensor, such as theimage-capture device or in different environmental or weatherconditions, for example, rain, hailstorm, during night, and the like.Although, an approximate distance of different objects in an image framemay be estimated by image processing, the distance or position ofobjects calculated from RF sensor or the LIDAR, may be faster and moreaccurate as compared to the image-processing methods. This helps toquickly and accurately generate the 3D digital model of the 3Dreal-world area surrounding the user 110 based on the sensor datareceived from the plurality of different types of sensors (such as theplurality of sensors 104).

The assistive device 102 may be configured to establish a mapping of theplurality of objects within the first proximity range to the pluralityof haptic elements of the haptic feedback interface 112, based on thereceived sensor data. In accordance with an embodiment, the generated 3Ddigital model may be utilized for the mapping of the plurality ofobjects within the first proximity range to the plurality of hapticelements.

In accordance with an embodiment, the assistive device 102 may beconfigured to generate a touch-discernible feedback on the hapticfeedback interface 112 based on the established mapping. Thetouch-discernible feedback may correspond to at least one of adifferential pressure-based, a differential temperature-based, adifferential electric pulse-based, a differential raised shapepattern-based touch-discernible feedback. In some embodiments, acombination of different touch-discernible feedback, for example, acombination of the differential electric pulse-based and thedifferential raised shape pattern-based touch-discernible feedback maybe employed. The touch-discernible feedback may comprise a plurality ofdifferential touch-discernible cues to discern a 3D arrangement of eachof the plurality of objects with respect to a position of the user 110of the assistive device 102. The 3D arrangement of each of the pluralityof objects may refer to an overall arrangement of the plurality ofobjects in a 3D physical space, such as the 3D real-world areasurrounding the user 110. As the plurality of differentialtouch-discernible cues also include a touch-discernible cue thatindicates the position of the user 110 in the generatedtouch-discernible feedback on the haptic feedback interface 112, the 3Darrangement of each of the plurality of objects from a perspective ofuser 110 may provide an indication to the user 110 as to where the user110 is currently present in the 3D real-world area with respect to or inrelation to other objects of the plurality of objects. It may beadvantageous to include at least one touch-discernible cue thatindicates the position of the user 110 in the generatedtouch-discernible feedback itself as it enables the user 110 to easilydiscern the 3D real-world area from the perspective of the user 110 by atouch on the differential touch-discernible cues. An exemplarytouch-discernible feedback and exemplary differential touch-discerniblecues are shown and described, for example, in FIG. 3C.

In some embodiments, the assistive device 102 may be configured tocontrol the output of an audio feedback via one or more audio-outputdevices provided in the assistive device 102. The audio feedback may beprovided in-sync with the generated touch-discernible feedback. Theaudio feedback may be generated as the user 110 moves from a firstlocation to a second location in the 3D real-world area within the firstproximity range. For example, as the user 110 moves from the firstlocation to a new location (such as the second location) in the 3Dreal-world area, the audio feedback in combination with thetouch-discernible feedback may provide an enhanced understanding of thenearby environment of the user 110 for navigation.

In accordance with an embodiment, the assistive device 102 may beconfigured to update the mapping of the plurality of objects to theplurality of haptic elements on the defined region of the hapticfeedback interface 112. The update may be done based on a change inposition of one or more movable objects of the plurality of objectsincluding the user 110. The assistive device 102 may be configured todetect the change in real time or near-real time from the receivedsensor data. For example, when one or more objects of the plurality ofobjects moves in the 3D real-world area, the generated 3D model at theassistive device 102, may also be updated. Thereafter, thetouch-discernible feedback generated previously may be dynamicallyupdated on the haptic feedback interface 112 based on the update of themapping. Thus, after an initial generation of the touch-discerniblefeedback, certain portion(s) of the haptic feedback interface 112 needsto be updated instead of entire haptic feedback interface 112. Thus, theupdate may be done quickly. In some embodiments, the update may occurperiodically. In some embodiments, the update may be done in real timeor near-real time continually as the one or more objects of theplurality of objects moves in the 3D real-world area. The updatedtouch-discernible feedback enables the user 110 to constantly discernchanges in the 3D real-world area surrounding the user 110.

FIG. 2 is a block diagram that illustrates an exemplary assistive devicefor non-visually discerning a 3D real-world area surrounding a user ofthe assistive device, in accordance with an embodiment of thedisclosure. FIG. 2 is explained in conjunction with elements fromFIG. 1. With reference to FIG. 2, there is shown the assistive device102. The assistive device 102 may include a processing section 202, asensor section 204, and a user interface section 206. The processingsection 202 may include a first circuitry 208, a second circuitry 210,and a memory 212. The sensor section 204 may include a microphone 214and a sensor cluster unit 216. The sensor cluster unit 216 may includeat least a biometric sensor 216A. The user interface section 206 mayinclude the haptic feedback interface 112, a haptic feedback controller220, and one or more audio-output devices, such as a first audio-outputdevice 224A and a second audio-output device 224B. The haptic feedbackinterface 112 may include a plurality of haptic elements 218. The hapticfeedback controller 220 may include a haptic feedback generator 222.

In accordance with an embodiment, the assistive device 102 may becommunicatively coupled to the plurality of sensors 104 through thefirst communication network 108A and/or the second communication network108B, by use of the first circuitry 208. The second circuitry 210 may becommunicatively coupled to the memory 212, and the various components ofthe sensor section 204 and the user interface section 206, via a systembus.

The first circuitry 208 may comprise suitable logic, circuitry,interfaces, and/or code that may be configured to receive sensor data ofthe 3D real-world area within a first proximity range of the assistivedevice 102. The sensor data of the 3D real-world area may be receivedfrom the plurality of sensors 104, via the first communication network108A. In some embodiments, the one or more sensors of the plurality ofsensors 104 may be provided as a part of the sensor cluster unit 216 asintegrated sensors. In such a case, the sensor data may be acquired bythe system bus for processing by the second circuitry 210. The firstcircuitry 208 may be further configured to communicate with externaldevices, such as the server 106, via the second communication network108B. The first circuitry 208 may implement known technologies tosupport wireless communication. The first circuitry 208 may include, butare not limited to, a transceiver (e.g. a radio frequency (RF)transceiver), an antenna, one or more amplifiers, a tuner, one or moreoscillators, a digital signal processor, a coder-decoder (CODEC)chipset, a subscriber identity module (SIM) card, and/or a local buffer.

The first circuitry 208 may communicate via wireless communication withnetworks, such as the Internet, an Intranet and/or a wireless network,such as a cellular telephone network, a wireless local area network(WLAN), a personal area network, and/or a metropolitan area network(MAN). The wireless communication may use any of a plurality ofcommunication standards, protocols and technologies, such as GlobalSystem for Mobile Communications (GSM), Enhanced Data GSM Environment(EDGE), wideband code division multiple access (W-CDMA), code divisionmultiple access (CDMA), LTE, time division multiple access (TDMA),Bluetooth, Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE802.11b, IEEE 802.11g, IEEE 802.11n, and/or any other IEEE 802.11protocol), voice over Internet Protocol (VoIP), Wi-MAX,Internet-of-Things (IoT) technology, Li-Fi, Machine-Type-Communication(MTC) technology, a protocol for email, instant messaging, and/or ShortMessage Service (SMS).

The second circuitry 210 may refer a digital signal processor (DSP). Thesecond circuitry 210 may comprise suitable logic, circuitry, interfaces,and/or code that may be configured to generate a 3D digital model of the3D real-world area within the first proximity range based on thereceived sensor data from the plurality of sensors 104 (or the sensorcluster unit 216). The generated 3D digital model may be mapped to theplurality of haptic elements 218 of the haptic feedback interface 112.The assistive device 102 may be a programmable device, where the secondcircuitry 210 may execute instructions stored in the memory 212. Otherimplementation examples of the second circuitry 210 may include, but arenot limited to a specialized DSP, a Reduced Instruction Set Computing(RISC) processor, an Application-Specific Integrated Circuit (ASIC)processor, a Complex Instruction Set Computing (CISC) processor, and/orother processors.

The memory 212 may comprise a learning engine. The second circuitry 210may be configured to determine one or more patterns in a plurality ofuser interactions on the haptic feedback interface 112 over a period oftime based on a track of a usage pattern of the assistive device 102 bythe learning engine. The memory 212 may include suitable logic,circuitry, and/or interfaces that may be configured to store a set ofinstructions executable by the second circuitry 210. The memory 212 maybe further configured to temporarily store one or more captured mediastreams, such as one or more videos or images of the 3D real-world areawithin the first proximity range (i.e. an area surrounding the user 110)as image buffer for processing by the second circuitry 210. The memory212 may also store usage history, an amount of pressure exerted by theuser 110 while touching the haptic feedback interface 112 in theplurality of user interactions on the haptic feedback interface 112 overa period of time. The memory 212 may also store input and outputpreference settings by the user 110. Examples of implementation of thememory 212 may include, but not limited to, a random access memory(RAM), a dynamic random access memory (DRAM), a static random accessmemory (SRAM), a thyristor random access memory (T-RAM), azero-capacitor random access memory (Z-RAM), a read only memory (ROM), ahard disk drive (HDD), a secure digital (SD) card, a flash drive, cachememory, and/or other non-volatile memory.

The microphone 214 may comprise suitable circuitry and/or interfaces toreceive an audio input. In accordance with an embodiment, the audioinput may be provided by the user 110. The audio input may correspond toa voice input to the assistive device 102. In accordance with anembodiment, the microphone 214 may be muted or disabled in accordancewith user preferences. Although FIG. 2 shows a single microphone, aperson of ordinary skill in the art may understand that the assistivedevice 102 may include multiple microphones to capture sound emanatingfrom the first proximity range of the user 110 of the assistive device102. In some embodiments, the assistive device 102 may include aplurality of microphones. Each microphone of the plurality ofmicrophones may be fitted at different locations of the assistive device102. Based on a difference in the time of receipt of a sound emanatedfrom an object of the plurality of objects in the 3D real-world area, ateach of microphone of the plurality of microphones, a direction of theobject may be determined. For example, four microphones may be placed atfour sides (left, right, top, and bottom) of the assistive device 102.In cases, a sound signal from an object, such as a human or vehiclehorn, may be received first at the left microphone, followed by frontmicrophone, and then at remaining microphones. This may indicate thatthe object may be located at 45-degree angle between left and frontdirection with respect to the current orientation of the assistivedevice 102. This information, such as the determined direction of theobject, may then be utilized during generation of the touch-discerniblefeedback or the audio feedback to discern the 3D arrangement of theplurality of objects in the 3D real-world area, as discussed in theFIGS. 2, 3A, 3B, 3C, and 3D.

The sensor cluster unit 216 may include a biometric sensor 216A, such asa fingerprint sensor, to decipher the identity of a user, such as theuser 110. In certain scenarios, the assistive device 102 may be used mymultiple users, for example, users of a same family. In such a case,based on user authentication by use of the biometric sensor, a differentusage profile and user settings may be loaded for different users. Insome embodiments, the sensor cluster unit 216 may also include atemperature sensor and a pressure sensor to gauge pressure applied by auser, such as the user 110, on the haptic feedback interface 112. Insome embodiments, one or more sensors of the plurality of sensors 104may be a part of the sensor cluster unit 216. For example, the sensorcluster unit 216 may include the location sensor, the image sensor, theRF sensor, the accelerometer, the gyroscope, the compass, themagnetometer, an integrated image-capture device, the depth sensor, thealtimeter, a lux meter, an ultrasound sensor, the IR sensor, or one ormore weather sensors.

The haptic feedback interface 112 may comprise the plurality of hapticelements 218.The plurality of haptic elements 218 may refer to an arrayof cylindrical tubes arranged at the surface of the haptic feedbackinterface 112. A person of ordinary skill in the art may understand thatshape of each tube may be variable, such as conical, hexagonal, or otherpolygonal shapes, without departing from the scope of the disclosure. Inaccordance with an embodiment, the plurality of haptic elements 218 maybe arranged as a layer (of array of cylindrical tubes) on the hapticfeedback generator 222 such that a haptic signal may be generated by thehaptic feedback generator 222 through each of the plurality of hapticelements 218. In accordance with an embodiment, one end (e.g. a proximalend) of each tube of the array of cylindrical tubes may be coupled tothe haptic feedback generator 222, and the other end (e.g. a distal end)may be interspersed on the haptic feedback interface 112 such that aplurality of differential touch-discernible cues generated by the hapticfeedback generator 222 in conjunction with the plurality of hapticelements 218 are discernible on the haptic feedback interface 112 by thesense of touch.

The haptic feedback controller 220 may comprise suitable circuitry andinterfaces to control output of a touch-discernible feedback on thehaptic feedback interface 112 by the haptic feedback generator 222. Thehaptic feedback controller 220 may be configured to sense a haptic userinput via plurality of haptic elements 218 based on a defined amount ofpressure detected at one or more haptic elements of the plurality ofhaptic elements 218. The haptic feedback controller 220 includes thehaptic feedback generator 222. The haptic feedback generator 222 maycomprise suitable circuitry and interfaces to receive mappinginformation from the second circuitry 210. The mapping informationcorresponds to the mapping of the 3D digital model that includes aplurality of objects of the 3D real-world area within the firstproximity range to the plurality of haptic elements 218. The hapticfeedback generator 222 may be configured to generate a touch-discerniblefeedback on the haptic feedback interface 112 based on the establishedmapping. The touch-discernible feedback comprises a plurality ofdifferential touch-discernible cues generated on the haptic feedbackinterface 112 by use of the plurality of haptic elements 218 to enablethe user 110 to non-visually discern the 3D real-world area surroundingthe user 110 within the first proximity range.

The one or more audio-output devices 224, such as the first audio-outputdevice 224A and the second audio-output device 224B, may comprisesuitable circuitry and/or interfaces to generate an audio output for theuser 110. In accordance with an embodiment, the audio output may begenerated in-sync with the touch-discernible feedback generated on thehaptic feedback interface 112. In accordance with an embodiment, theaudio output may be generated in-sync with a haptic input received onthe haptic feedback interface 112 for multi-sense discern of thetouch-discernible feedback for enhanced understanding of the surroundingof the user 110. The haptic input may be detected by the haptic feedbackcontroller 220 by use of the pressure sensor of the sensor cluster unit216.In accordance with an embodiment, the one or more audio-outputdevices 224 may be muted or disabled based on a time-of-day or for aspecific location, such as a public library where silence is solicited.Though FIG. 2 is shown to include two audio-input devices, a person ofordinary skill in the art may understand that the assistive device 102may include a single audio-input device, or more than two audio-inputdevices. The other speakers may be placed at corners, for example, atextreme left and right corners of the assistive device 102, to aid invoice-based navigation of the user 110 as the user 110 moves with theassistive device 102 from one location to another location in the 3Dreal-world area. In some embodiments, one or more audio-input devicesmay be provided or worn at different parts of the body of the user 110for voice-based navigation of the user 110 as the user 110 moves withthe assistive device 102 from one location to another location in the 3Dreal-world area. Such voice-based navigation may be provided incombination to the generated touch-discernible feedback, which may actsynergistically to provide enhanced navigation assistance to the user110 in real time or near-real time as the user 110 moves in the 3Dreal-world area. An example is described below in FIG. 2 and FIG. 3C.

In operation, the second circuitry 210 may be configured to detect acurrent location of the assistive device 102, by use of the locationsensor. As the user 110 may be equipped with the assistive device 102,the location of the assistive device 102 may be same as that of the user110. The location sensor may be an integrated sensor of the assistivedevice 102 provided in the sensor cluster unit 216 or may be one of theplurality of sensors 104. The assistive device 102 may be a hand-helddevice or a wearable device. The second circuitry 210 may be configuredto check whether a first template map of a 3D real-world area for thedetected current location of the assistive device 102, is available. Insome embodiments, where the first template map of the 3D real-world areais available, the first circuitry 208 may be configured to acquire thefirst template map of the 3D real-world area within the first proximityrange (e.g. the first proximity range 304) of the assistive device102.The first template map may be acquired from the server 106 based onthe current location of the assistive device 102. In some embodiments,the memory 212 may store 2D/3D maps of geographical regions of the earthsurface, such as street views. In such a case, the second circuitry 210may be configured to retrieve the first template map of the 3Dreal-world area from the memory 212. The first template map may beavailable for certain outdoor areas, whereas such maps may not beavailable for indoor areas.

In accordance with an embodiment, the first circuitry 208 may beconfigured to receive sensor data of the 3D real-world area within thefirst proximity range of the assistive device 102 from the plurality ofsensors 104 that are communicatively coupled to the assistive device102. In some embodiments, the sensor data may also be received from thesensor cluster unit 216. In some embodiments, the first template map ofa 3D real-world area may not be acquired, for example, in case of indoorlocations or for regions where the first template map may not beavailable. In such a case, the sensor data of the 3D real-world areareceived in real time or near-real time may be used to collectinformation of the 3D real-world area within the first proximity rangeof the assistive device 102.

In accordance with an embodiment, the second circuitry 210 may befurther configured to identify the object-type of each of the pluralityof different objects present within the first proximity range of theassistive device 102 based on the received sensor data. The secondcircuitry 210 may be configured to determine a relative position of eachof the plurality of objects with respect to the position of the user 110of the assistive device 102. The relative position of each of theplurality of objects may be determined based on the sensor data receivedin real time or near-real time from the plurality of sensors 104 worn bythe user 110. The second circuitry 210 may be configured to determine aheight of each of the plurality of objects from the perspective of theheight of the user 110 of the assistive device 102.

In accordance with an embodiment, the second circuitry 210 may beconfigured to update the first template map in real time or near-realtime based on the sensor data of the 3D real-world area. The secondcircuitry 210 may be configured to generate a 3D digital model of the 3Dreal-world area, based on the received sensor data. The second circuitry210 may be further configured to compute a scaling factor in accordancewith the first proximity range and a surface area of the haptic feedbackinterface 112 or a surface area of a defined region of the hapticfeedback interface 112. The defined region of the haptic feedbackinterface 112 may refer to an overall surface area of the hapticfeedback interface 112 on which a haptic feedback is to be generated. Anexample of the defined region (e.g. the defined region 322) of thehaptic feedback interface 112 is shown in FIG. 3C and 3D.

In accordance with an embodiment, the second circuitry 210 may beconfigured to establish a mapping of the plurality of objects to theplurality of haptic elements 218 in the defined region of the hapticfeedback interface 112, based on the received sensor data. In accordancewith an embodiment, the generated 3D digital model may be utilized forthe mapping of the plurality of objects within the first proximity rangeto the plurality of haptic elements 218.

In accordance with an embodiment, the haptic feedback generator 222 maybe configured to generate a touch-discernible feedback on the hapticfeedback interface 112 based on the established mapping. Thetouch-discernible feedback may comprise a plurality of differentialtouch-discernible cues to discern a 3D arrangement of each of theplurality of objects with respect to a position of the user 110 of theassistive device 102. The haptic feedback generator 222 generates theplurality of differential touch-discernible cues such that the user 110can discern different identified object-types of the plurality ofobjects present within the first proximity range of the assistive device102.

The plurality of differential touch-discernible cues refers totouch-discernible cues generated on the haptic feedback interface thatmay be dissimilar or different in one or more physical characteristicsor properties such that a user may discern their disparate form bytacioperception. The one or more physical characteristics or properties,for example, may be shape, size, smoothness, roughness, temperature,amount of pressure, pain, level of elevation, a pattern of distribution,and the like, which may be discernible by touch. In accordance with anembodiment, the plurality of differential touch-discernible cues may begenerated as a plurality of protrusions of different shapes that areextended from the surface of the haptic feedback interface 112. Eachprotrusion may be a raised shape-pattern or a bulge that sticks out fromat least one or a group of haptic elements of the plurality of hapticelements 218 of the haptic feedback interface 112. The plurality ofprotrusions represents the plurality of objects of the 3D real-worldarea within the first proximity range. One shape may be assigned to oneidentified object-type of the plurality of objects of the 3D real-worldarea within the first proximity range to enable the user 110 to discernthe object-type when the user 110 touches a protrusion of a definedshape. For example, an oval shape protrusion may denote a particularobject-type, for example, a car. Examples of the oval shape protrusionsmay be the touch-discernible cues 324 c and 324 d, as shown in FIG. 3C.A round protrusion may denote a human being. Examples of the roundprotrusion may be the touch-discernible cues 324 a and 324 b, as shownin FIG. 3C. A square-shaped protrusion may denote a building, and apole-like or a spike-like protrusion may denote a pillar or a pole inthe 3D real-world area within the first proximity range. Examples of thesquare-shaped protrusions may be the touch-discernible cues 324 f, 324g, and 324 h, as shown in FIG. 3C. An example of the pole-like or aspike-like protrusion may be the touch-discernible cue 324 e as shown inFIG. 3C. Thus, when the user 110 touches the oval shape protrusion, theuser 110 may readily identify the protrusion to be a car. Thus, similarto the sighted people who use information about the features on thesurface of an object, like color, shading, or overall size, and shape,to recognize an object, the people who have lost the sense of sight mayalso identity an object based on a touch on the protrusion of a definedshape, where an association of a particular shape with a particularobject-type is learned by brain.

In accordance with an embodiment, the plurality of protrusions generatedon the haptic feedback interface 112 enables the user 110 to discern notonly the object-type but also a relative positioning of the plurality ofobjects from the perspective of the user 110. Further, as the pluralityof objects are mapped to the plurality of the haptic elements 218 of thehaptic feedback interface 112, a 3D arrangement of the plurality ofobjects is discernible by touch on the plurality of differentialtouch-discernible cues that are generated as the plurality ofprotrusions of different shapes extended from the surface of the hapticfeedback interface 112. The 3D arrangement may indicate not only theposition or distance of each object of the plurality of objects from theuser 110 of the assistive device 102, but also may indicate a relativesize or direction of travel of objects for moving objects.

In accordance with an embodiment, the plurality of protrusions may be ofsame shapes. In such a case, although it may be relatively difficult toidentity an object-type, however, the relative position of each of theplurality of objects from the position of the user 110 may be easilydiscernible by touch on the plurality of protrusions. Further, as theuser 110 is present in the 3D real-world area, the user 110 may hearactual sound emanated from one or more objects of the plurality ofobjects. Hence, the user 110 may correlate the plurality of protrusionswith the plurality of sounds to discern an object-type or estimate anapproximate distance to an object of the plurality of objects. Thehaptic feedback generator 222 may be configured to control the extendingand the retracting of the plurality of protrusions by use of theplurality of haptic elements 218. The haptic feedback generator 222 maybe configured to control grouping of the plurality of haptic elements218 during extension to represent a particular shape for a protrusion.In accordance with an embodiment, the protrusion may be static or may bedeformable. The same protrusion may have different meanings based on thedeformation. For example, a 3D real-world area surrounding the user 110may include a sportsman in a play ground. The sportsman while playing asoccer game may be standing on the playground or may be walking, andsuddenly fell down. In such as a case, the protrusion (that mayrepresent the sportsman) may be deformed to represent a sudden changefor the same object in the 3D real-world area. The sudden change may bediscernible by the user 110 in the generated touch-discernible feedbackfor the playground, based on the touch-discernible cue of the deformingprotrusion. In some embodiments, the user 110 may be present on a riverside and use the assistive device 102 to generate a touch-discerniblefeedback of the 3D real world area surrounding the user 110. A certainraised shape pattern generated on the haptic feedback interface 112 mayrepresent water body (e.g. a river) ahead of the user 110. The raisedshape pattern may be a constantly deforming protrusion. Based on a touchon the constantly deforming protrusion, the user 110 may discern notonly a presence of a water-body, such as the river, based on a touch onthe constantly deforming protrusion but also an exact location of theriver, and the relative position of the user 110 from the water body inthe generated touch-discernible feedback.

In accordance with an embodiment, the plurality of protrusions may begenerated by application of different temperatures on different surfacearea of the haptic feedback interface 112. In such an embodiment, thehaptic feedback interface 112 may include a covering on the uppersurface (such as the upper surface 102a (FIG. 3A)). The covering may bea polymer-based layer sensitive to temperature. The plurality of thehaptic elements 218 may be arranged as the array of cylindrical tubesbelow the covering. In cases where, a localized high temperature isgenerated through one or a group of the haptic elements of the pluralityof haptic elements 218, a bulge may appear on the covering of the hapticfeedback interface 112. Similarly, different bulge portions mayrepresent the plurality of protrusions. Notwithstanding, the pluralityof protrusions may be generated by various methods, such as byelectro-chemical process, without limiting the scope of the disclosure.In accordance with an embodiment, the plurality of differentialtouch-discernible cues may be generated as different level ofelectric-pulses or a different amount of pressure, such as pain points(or pricking points) that may represent the positioning of the pluralityof objects of the 3D real world area in the generated touch-discerniblehaptic feedback.

In accordance with an embodiment, the plurality of differentialtouch-discernible cues may be generated as multi-level pressure on thehaptic feedback interface 112 by the haptic feedback generator 222. Forexample, a first object of the plurality of objects in the 3D real-worldarea may be discernible by generating a haptic signal through one ormore haptic elements of the plurality of haptic elements 218 as a firstamount of pressure. This first amount of pressure may be felt by theuser 110 when the user 110 touches a specific portion, for example, afirst portion, of the haptic feedback interface 112. Similarly, for eachposition of different objects of the plurality of objects, a differentamount of pressure may be generated on the haptic feedback interface112. Thus, the user 110 may feel different amount of pressure atdifferent points (or portions) on the haptic feedback interface 112. Thedifferent amount of pressure enables the user 110 (by touch on thehaptic feedback interface 112) to non-visually discern the relativepositioning of the plurality of objects of the 3D real world area. Thedifferent amount of pressure in the generated touch-discernible hapticfeedback corresponds to the plurality of differential touch-discerniblecues generated as multi-level pressure.

In accordance with an embodiment, the plurality of differentialtouch-discernible cues may be generated as different temperatures, forexample, different combination of hot and cold temperatures, on thehaptic feedback interface 112 by the haptic feedback generator 222. Foreach position of different objects of the plurality of objects, adifferent temperature level may be generated on the haptic feedbackinterface 112 through one or more haptic elements of the plurality ofhaptic elements 218. The different level of temperature enables the user110 (by touch on the haptic feedback interface 112) to non-visuallydiscern the relative positioning of the plurality of objects includingthe user 110 in the 3D real world area.

In accordance with an embodiment, the plurality of differentialtouch-discernible cues may be generated as different level ofelectric-pulses on the haptic feedback interface 112 by the hapticfeedback generator 222. For each position of different objects of theplurality of objects, a different level of electric-pulse may begenerated on the haptic feedback interface 112 through a haptic elementof the plurality of haptic elements 218. The different level ofelectric-pulses enables the user 110 (by touch on the haptic feedbackinterface 112) to non-visually discern the relative positioning of theplurality of objects of the 3D real world area. The different amount ofelectric-pulses in the generated touch-discernible haptic feedbackcorresponds to the plurality of differential touch-discernible cuesgenerated as different level of electric-pulses. Further, when an objectof the plurality of objects moves in the 3D real-world area, anelectric-pulse (i.e. a touch-discernible cue) may also felt on thehaptic feedback interface 122 to be moving as a continuous line from onepoint of the haptic feedback interface 122 to another point to representthe movement and a direction of movement of the object of the pluralityof objects in the 3D real-world area. The generation of electric-pulse(i.e. a touch-discernible cue) along a certain path on the hapticfeedback interface 122 may be synchronized to the actual movement of theobject in the 3D real-world area. This allows the user 110 to understandthe path of movement of the object simply by placing one hand on adefined region of the haptic feedback interface 112, as shown in FIG.3D. In case of the assistive device 102 is a wearable device, similartouch-discernible cues (e.g. different amount of pressure, differentlevel of electric-pulses, different temperatures (such as hold andcold), different shape patterns, static or deformable protrusions,movement of touch-discernible cues), may be felt based on the contact ofthe skin of the user 110 with the haptic feedback interface 112 that maybe wrapped on a body part, such as waist, or arm, as a wrap band, orworn as a pad. The movement of a touch-discernible cue, for example, aparticular electric-pulse running from one point to another point of thehaptic feedback interface 112, may further indicate a movement of anobject of the plurality of objects in the 3D real-world area in thefirst proximity range of the user 110.

In accordance with an embodiment, the second circuitry 210 may beconfigured to control the output of the audio feedback via the one ormore audio-output devices 224 as the user 110 moves from a firstlocation to a second location in the 3D real-world area within the firstproximity range. In accordance with an embodiment, the haptic feedbackinterface 112 may receive an input on the same surface of the hapticfeedback interface 112 where the touch-discernible feedback isgenerated. For example, the user 110 may press a protrusion (or a bulge)generated on the haptic feedback interface 112. Based on the amount ofpressure exerted by the user 110 while touching the protrusion on thehaptic feedback interface 112, the press may be considered a hapticinput by the haptic feedback controller 220. In cases where the amountof pressure exerted by the user 110 on a particular point or aprotrusion on the haptic feedback interface 112 is greater than athreshold pressure value, the press of the protrusion (or a bulge) maybe considered a haptic input for that particular object of the 3Dreal-world area that is indicated by the pressed protrusion. Acorresponding action related to the pressed protrusion may be executedby the haptic feedback controller 220 in association with the secondcircuitry 210. For example, an oval shape protrusion, which denotes aparticular object-type, for example, a car, may be pressed. An audiofeedback for that car may be generated. For example, “this is a car at adistance of 3 meters from you, be careful”. Such voice-based feedbackprovided in combination to the generated touch-discernible feedbackprovide a synergistic and enhanced non-visual navigation assistance tothe user 110 in real time or near-real time as the user 110 moves in the3D real-world area. For example, in this case the input detected on theoval shape protrusion for car is augmented with voice feedback thatprovides additional information for that particular car. Further, inconventional devices, the input section to receive a haptic input isdifferent from the output section (in a conventional haptic userinterface) where the Braille output or other tactile forms of output aregenerated. Typically, input section to receive haptic input is a 6-keysor 8-keys Braille input. A separate section to receive input and provideoutput, may be considered a rudimentary form of HMI, where a generatedhaptic output may not be capable of receive a further feedback on aparticular touch-discernible cue. In contrast, the same tactile surfacearea of haptic feedback interface 112 of the assistive device 102 actsboth as the haptic input receiver and haptic output generator, where theuser 110 may press a protrusion (or a bulge) generated on the hapticfeedback interface 112 to provide the haptic input related to a specificobject in the vicinity of the assistive device 102. Based on the amountof pressure exerted by the user 110 while touching the protrusion on thehaptic feedback interface 112, the press may be considered a hapticinput by the haptic feedback controller 220.

In accordance with an embodiment, the haptic feedback generator 222 maybe configured to update the mapping of the plurality of objects to theplurality of haptic elements 218 in the defined region of the hapticfeedback interface 112. The update may be done based on a change inposition of one or more movable objects of the plurality of objectsincluding the user 110. The second circuitry 210 may be configured todetect the change in real time or near-real time from the receivedsensor data.

In accordance with an embodiment, the haptic feedback generator 222 maybe configured to dynamically update the touch-discernible feedback onthe haptic feedback interface 112 based on the update of the mapping, inconjunction with the second circuitry 210 and the haptic feedbackcontroller 220. The updated touch-discernible feedback comprises achange in the relative positions of one or more of the plurality ofobjects with respect to the position of the user 110 of the assistivedevice 102.

FIGS. 3A, 3B, 3C, and 3D illustrate exemplary scenario diagrams forimplementation of the assistive device and method for non-visuallydiscerning a 3D real-world area surrounding a user, in accordance withan embodiment of the disclosure. With reference to FIG. 3A, there is ashown a first exemplary scenario 300A, which is described in conjunctionwith elements from FIGS. 1 and 2. The first exemplary scenario 300Ashows the user 110 with the assistive device 102 present in a 3Dreal-world area. There is also shown the microphone 214, the firstaudio-output device 224A, the second audio-output device 224B, and theplurality of haptic elements 218 of the haptic feedback interface 112.An image-capture device 302 may be placed worn by the user 110 of theassistive device 102, for example, as a headset. There is also shown afirst proximity range 304 of the assistive device 102. The firstproximity range 304 includes a certain portion (or sub-area) of the 3Dreal-world area surrounding the user 110 of the assistive device 102.There is also shown an upper surface 102a and a lower surface 102 b ofthe assistive device 102.

In accordance to the first exemplary scenario 300A, the user 110 may bea person with loss or impaired sight. The haptic feedback interface 112is shown in a first state before the generation of any haptictouch-discernible feedback on the haptic feedback interface 112. Theassistive device 102 may receive a voice input from the user 110 via themicrophone 214. In accordance with an embodiment, the first audio-outputdevice 224A and the second audio-output device 224B may output audiofeedback to aid in voice-based navigation of the user 110 as the user110 moves with the assistive device 102 from one location to anotherlocation in the 3D real-world area. In some embodiments, the lowersurface 102 b of the assistive device 102 may include a holding strap(not shown) in which one hand of the user 110 may be inserted so as toenable the user 110 to hold the assistive device 102 using the one handwhile the other hand is free to be placed on the upper surface 102 a ofthe assistive device 102 on the haptic feedback interface 112. Inaccordance with an embodiment, the assistive device 102 may include aplurality of hardware control buttons (not shown), such as a powerbutton to ON/OFF the assistive device 102, a reset button to reset thegenerated touch-discernible feedback 320 (FIG. 3C) on the hapticfeedback interface 112 to the first state, one or more volume controlbuttons/wheels to control audio output from the first audio-outputdevice 224A and the second audio-output device 224B, a mute button todisable audio output, a proximity range setter to set a proximity range,and the like. The assistive device 102 may include various othercomponents, as shown in FIG. 2, but for the sake of brevity are excludedfrom FIG. 3A.

With reference to FIG. 3B, there is shown a second exemplary scenario300B that depicts the 3D-real world area surrounding the user 110 withinthe first proximity range 304 (of FIG. 3A) of the assistive device 102.FIG. 3B is explained in conjunction with elements from FIGS. 1, 2, and3A. In the second exemplary scenario 300B, the 3D-real world areasurrounding the user 110 within the first proximity range 304 includes aplurality of different objects. The plurality of different objects mayinclude both moving objects (e.g. the user 110, another person 306, afirst car 308, and a second car 310), and stationary objects (e.g. apole 312, a plurality of buildings 314, 315, and 316, and a four-waystreet intersection 318. The 3D-real world area surrounding the user 110within the first proximity range 304 may include many other objects,such as street lights, and the like, which are not shown for the sake ofbrevity.

In accordance with the second exemplary scenario 300B, the user 110 maybe holding the assistive device 102. The user 110 may press a power “ON”button to initiate receipt of sensor data from the plurality of sensors104. For example, the image-capture device 302 may be worn as a headsetor placed at a suitable position on the body of the user 110 to capturea 360 view of the 3D real-world area that surrounds the user 110 withina first proximity range, for example, “X” meters, where “X” refers to adistance in natural numbers. In accordance with the second exemplaryscenario 300B, the first proximity range may be set as 40 meters. Theproximity range setter wheel provided in the assistive device 102 may beused to set the desired proximity range by the user 110. In someembodiments, the first proximity range may be a user-specified defaultrange. In some embodiments, the first proximity range may correspond toan equal ‘X” meters range from the center that corresponds to theposition of the user 110. In some embodiments, the first proximity rangemay correspond to an unequal ‘X” meters range from the position of theuser 110, for example, more area may be covered in front, left, andright of the user 110 based on a direction of movement of the user 110as compared to the rear area of the user 110.

In accordance with an embodiment, the first circuitry 208 may beconfigured to receive sensor data of the 3D real-world area within thefirst proximity range 304 of the assistive device 102. The sensor datamay include the captured 360-degree view of the 3D real-world area thatsurrounds the user 110 within the first proximity range and RF sensordata that provide an estimation of distances and motion of each theplurality of different objects from the position of the user 110. Thesensor data may also include sensed data from the IR sensor of theplurality of sensors 104. The sensed data from the IR sensor may be usedto distinguish between living and non-living objects. The sensor data ofthe 3D real-world area within the first proximity range 304 may bereceived from the plurality of sensors 104. The plurality of sensors 104may include wearable sensors that may be worn by the user 110, sensorsthat may be integrated with the assistive device 102, such as sensors ofthe sensor cluster unit 216, or sensors provided in other personaldevices of the user 110. The sensor data of the 3D real-world areareceived in real time or near-real time may be used to collectinformation of the 3D real-world area within the first proximity range304 of the user 110.

In accordance with an embodiment, the second circuitry 210 may beconfigured to generate a 3D digital model of the 3D real-world area,based on the received sensor data. The 3D digital model may correspondto a digital reproduction of the 3D real-world area within the firstproximity range 304. The 3D digital model may include positionalinformation of the plurality of different objects, such as the movingobjects (e.g. the user 110, the person 306, the first car 308, and thesecond car 310), and stationary objects (e.g. the pole 312, theplurality of buildings 314, 315, and 316, and the streets of thefour-way street intersection 318. The 3D digital model may include arelative position (or relative distances) of each of the plurality ofdifferent objects with respect to the position of the user 110 of theassistive device 102. The 3D digital model may include heightinformation of each of the plurality of different objects with respectto the height of the user 110. The 3D digital model may also include anobject-type identifier assigned to each object of the plurality ofdifferent objects. A same object-type identifier may be assigned to sameor similar types of objects. For example, all human beings present inthe first proximity range 304 may be assigned a same object-typeidentifier (e.g. first object-type identifier). Similarly, all vehiclespresent in the first proximity range 304 may be assigned a sameobject-type identifier but different from the object-type identifierassigned to the human beings (e.g. second object-type identifier). Inaccordance with an embodiment, the 3D digital model may also includesize, and texture information of each of the plurality of differentobjects present in the first proximity range 304. As the 3D digitalmodel includes information of the moving objects and the stationaryobjects, moving objects, for example moving cars may beidentified/distinguished from stationary cars.

In accordance with an embodiment, the second circuitry 210 may beconfigured to compute a scaling factor based on the first proximityrange 304 selected on the assistive device 102 and a defined region(such as the defined region 322 (FIG. 3C)) of the haptic feedbackinterface 112 on which a haptic feedback is to be mapped. The scalingfactor denotes how much reduction in size, and relative adjustment ofthe size, shape, height, and position of the plurality of differentobjects may be required to be able to map the plurality of differentobjects to the plurality of haptic elements 218 in the defined region ofthe haptic feedback interface 112. In accordance with an embodiment, thesecond circuitry 210 may be configured to establish the mapping of theplurality of different objects to the plurality of haptic elements 218in the defined region of the haptic feedback interface 112, based on thereceived sensor data and the generated 3D digital model.

With reference to FIG. 3C, there is shown a third exemplary scenario300C that depicts a touch-discernible feedback 320 generated on adefined region 322 of the haptic feedback interface 112. FIG. 3C isexplained in conjunction with elements from FIG. 1, 2, 3A, and 3B. Thetouch-discernible feedback 320 includes a plurality of differentialtouch-discernible cues 324 a, 324 b, 324 c, 324 d, 324 e, 324 f, 324 g,324 h, and 324 i. The plurality of differential touch-discernible cues322 a to 322 i represents the plurality of different objects in the 3Dreal-world area within the first proximity range 304 of the assistivedevice 102.

In accordance with an embodiment, the haptic feedback generator 222 maybe configured to generate the touch-discernible feedback 320 on thehaptic feedback interface 112 based on the established mapping. Thetouch-discernible feedback 320 may comprise a plurality of differentialtouch-discernible cues 324 a to 324 i to discern a 3D arrangement ofeach of the plurality of different objects with respect to a position ofthe user 110 of the assistive device 102. For example, the plurality ofdifferential touch-discernible cues 324 a to 324 i may be generated asthe plurality of protrusions of different shapes that are extended fromthe surface of the haptic feedback interface 112. One shape may beassigned to one identified object-type of the plurality of objects ofthe 3D real-world area within the first proximity range to enable theuser 110 to discern the object-type when the user 110 touches aprotrusion of a defined shape. For example, the touch-discernible cue324 a may be a round protrusion that indicates the position of the user110 in the 3D real-world area. The touch-discernible cue 324 b may be around protrusion that indicates the object-type as human being and theposition of the person 306 (FIG. 3B) in the 3D real-world area. Thetouch-discernible cue 324 c and 324 d may be oval protrusions thatindicate the object-type as car and the positions of the first car 308and the second car 310 respectively. The touch-discernible cue 324 c and324 d also indicates the direction of travel discernible by touch. Thetouch-discernible cue 324 e may be a spike-shaped protrusion thatindicates the object-type as pole and the position of the pole 312 inthe 3D real-world area. Similarly, the touch-discernible cues 324 f, 324g, and 324 h may be square-shaped protrusions that indicate theobject-type as buildings and the positions of the buildings 314, 315,and 316 respectively. Lastly, the touch-discernible cue 324 i may be afirst set of raised parallel lines and a second set of raised parallellines crossing each other at the centre that indicate the object-type as“streets intersection” and the positions and layout of the streets ofthe four-way street intersection 318 (FIG. 3B). Thus, similar to thesighted people who use information about the features on the surface ofan object, like color, shading, or overall size, and shape, to recognizean object, the people who have lost the sense of sight may also identityan object-type and object position based on a touch on the protrusion ofa defined shape in the generated touch-discernible feedback 320, wherean association of a particular shape with a particular object-type islearned by brain. The user 110 may discern the 3D real-world areasurrounding the user 110 within the first proximity range 304 based onthe generated touch-discernible haptic feedback 320.

In accordance with an embodiment, the second circuitry 210 may beconfigured to control the output of an audio feedback via the one ormore audio-output devices 224 as the user 110 moves in the 3D real-worldarea within the first proximity range 304. For example, based on thecurrent position of the user 110 (indicated by the touch-discernible cue324 a), an audio feedback may be generated as “There is a pole nearby 4feet on your right that have a push-to-walk button. Please locate andpress the push-to-walk button on the pole 312 to activate the pedestrianwalking signal to safely cross the road”. Further, the audio feedbackvia the one or more audio-output devices 224 may provide navigationalassistance, for example, turn left, turn right, cross the street, stophere, start moving, and the like, in combination with the generatedtouch-discernible feedback 320. Such voice-based feedback provided incombination to the generated touch-discernible feedback 320 provide asynergistic and enhanced non-visual navigation assistance to the user110 in real time or near-real time as the user 110 moves in the 3Dreal-world area. In some embodiments, the voice-based feedback may becustomized output for the user 110, for example, the user 110 may wantto reach to the destination, for example, a bank (i.e. the building 314(FIG. 3B)) from the current location near the pole 312. Thus, customizedvoice feedbacks may be generated in sequence to provide non-visualnavigation assistance to the user 110, for example, turn right and move4 steps, then turn left and move 5 steps, press the push-to-walk buttonon the pole 312 on your right, wait for 60 seconds for the pedestrianwalk signal, walk signal is now activated, start moving ahead for 30steps, turn left and move ahead for 10 steps, the bank is on your right.Turn right and move 5 steps to enter the building 314. Thus, based onthe learning engine and usage history, the second circuitry 210 may beconfigured to convert the distance to user steps to enable the user 110to readily and easily understand the instructions in the customizedaudio feedback in combination with the generated touch discerniblefeedback 320.

With reference to FIG. 3D, there is shown a fourth exemplary scenario300D that depicts an exemplary placement of a hand of the user 110 on adefined region 322 of the haptic feedback interface 112 for non-visuallydiscerning the 3D real-world area surrounding the user 110 within thefirst proximity range 304. It is known that the sense of touch has amuch greater sensory resolution than the sense of sight. Hence, thesense of touch can detect even small changes on a surface of the hapticfeedback interface 112. The user 110 may non-visually discern the 3Dreal-world area surrounding the user 110 within the first proximityrange 304 by tactioception based on a user touch on thetouch-discernible feedback 320 in the defined region 322 of the hapticfeedback interface 112. The haptic feedback generator 222 may beconfigured to dynamically update the touch-discernible feedback 320 andthe positions of the one or more touch-discernible cues on the hapticfeedback interface 112. The update may be done based on a change inposition of one or more movable objects, such as the first car 308, thesecond car 310, the person 306, and the user 110. The second circuitry210 may be configured to detect the change in real time or near-realtime from the received sensor data and signal the changes to the hapticfeedback generator 222 to update the touch-discernible feedback 320.

In some embodiments, the assistive device 102 may be implemented as ahand-held portable device. In some embodiments, the assistive device 102may be implemented as one or more wearable devices that may be wornaround at different parts of the human body having sensory receptorsthat senses touch. In such embodiments, the haptic feedback interface112 may be a foldable or bendable layer of pad or wrap band that may beworn on different parts of the body a user, such as the user 110. Theassistive device 102 may also include pads, bands, or straps, to enablethe assistive device 102 to be worn at different parts of the body ofthe user 110. For example, the assistive device 102 may be implementedas specialized hand gloves, where multiple haptic feedback interfaces(similar to the haptic feedback interface 112) may be in contact withskin of both the upper side and lower side (i.e. palm) of one or bothhands of the user 110 to convey information of the 3D real-world area inthe form of the touch-discernible haptic feedback generated by thehaptic feedback generator 222.

In one example, the assistive device 102 with the haptic feedbackinterface 112 may be worn as a shoe sole that provides touch-discerniblehaptic feedback. In some embodiments, multiple assistive devices withthe haptic feedback interface 112 may be worn, for example, one as ashoe sole and other as a hand-held device. In another example, theassistive device 102 with the haptic feedback interface 112 may bewrapped around one or both forearms of a human body, such as the user110. Thus, similar to the hand-held device, when the assistive device102 is worn, the skin of the user 110 (e.g. sensory receptors at skin ofthe forearms, thigh, waist, leg, feet, and the like) may feel theplurality of differential touch-discernible cues 324 a to 324 i in thetouch-discernible feedback 320 without a touch by a hand or finger(s) ofhand for non-visually discerning the 3D real-world area surrounding theuser 110 within the first proximity range 304. In FIGS. 3C and 3D, theplurality of differential touch-discernible cues, for example, are shownto be generated as a plurality of different protrusions of differentshapes. However, the plurality of differential touch-discernible cuesmay also be generated as different level of electric-pulses, differentamount of pressure or pain, different level of temperature, or theircombination, on the haptic feedback interface 112 by the haptic feedbackgenerator 222, as described in FIG. 2.

In accordance with an embodiment, the assistive device 102 may include aview-change button. The view-change button may be used by the user 110to change the capture of sensor data for a front area of the 3D-realworld area instead of all the area within the first proximity range 304.Thereby, the touch-discernible feedback may be generated for the frontarea of the 3D-real world area (i.e. a front view from the perspectiveof user 110). Similarly, a second press on the view-change button mayresult in the generation of the touch-discernible feedback for rearview, for example, to view an area behind the user 110. In someembodiments, the haptic feedback interface 112 may comprise a pluralityof defined regions, for example, two defined regions. A first definedregion of the plurality of defined regions may be configured to generatea first touch-discernible feedback for the front view, whereas a seconddefined region of the plurality of defined regions may be configured togenerate a second first touch-discernible feedback for the rear viewfrom the perspective of the user 110. In some embodiments, the modalityof generation of the plurality of differential touch-discernible cuesfor the first touch-discernible feedback may be same as the second firsttouch-discernible feedback. In some embodiments, the modality ofgeneration of the plurality of differential touch-discernible cues forthe first touch-discernible feedback may be different from the secondfirst touch-discernible feedback. The modality of generation of theplurality of differential touch-discernible cues corresponds togeneration of the plurality of differential touch-discernible cues asdifferent protrusions of different shapes, different level ofelectric-pulses, different amount of pressure or pain, different levelof temperature, or their combination, on the haptic feedback interface112.

FIGS. 4A and 4B, collectively, depict a flow chart 400 that illustratesa method for non-visually discerning a 3D real-world area surrounding auser, in accordance with an embodiment of the disclosure. FIGS. 4A and4B are described in conjunction with elements from the FIGS. 1, 2, and3A to 3D. As shown in FIG. 4A, the method of the flow chart 400 startsat 402 and proceeds to 404.

At 404, a current location of the assistive device 102 may be detected.The second circuitry 210 may be configured to detect the currentlocation of the assistive device 102 using the location sensor. Thelocation sensor may be provided in the sensor cluster unit 216 of theassistive device 102 or may refer to one of the plurality of sensors104. At 406, it may be checked whether a first template map of a 3Dreal-world area for the detected current location of the assistivedevice 102 is available. The availability of the first template map of a3D real-world area may be checked at the server 106 or the memory 212.In cases where the first template map is available, the control passesto 408, else to 410.

At 408, a first template map of a 3D real-world area within a firstproximity range of the assistive device 102 may be acquired. The firstcircuitry 208 may be configured to acquire the first template map of the3D real-world area within the first proximity range (e.g. the firstproximity range 304) of the assistive device 102. In accordance with anembodiment, the first template map may be acquired from the server 106based on the current location of the assistive device 102. As the user110 may be equipped with the assistive device 102, the location of theassistive device 102 may be same as that of the user 110. In someembodiments, the memory 212 may store 2D/3D maps of geographical regionsof the earth surface, such as street views.

At 410, sensor data of the 3D real-world area within the first proximityrange of the assistive device 102 may be received. The first circuitry208 may be configured to receive sensor data of the 3D real-world areawithin the first proximity range (e.g. the first proximity range 304) ofthe assistive device 102 from the plurality of sensors 104 that arecommunicatively coupled to the assistive device 102. In someembodiments, the sensor data may also be received from the sensorcluster unit 216. In some embodiments, the first template map of a 3Dreal-world area may not be acquired, for example, in case of indoorlocations or for regions where the first template map may not beavailable. In such a case, the sensor data of the 3D real-world areareceived in real time or near-real time may be used to collectinformation of the 3D real-world area within the first proximity rangeof the assistive device 102.

At 412, an object-type of each of the plurality of objects presentwithin the first proximity range of the assistive device 102 may beidentified, based on the received sensor data. The second circuitry 210may be further configured to identify the object-type of each of theplurality of different objects present within the first proximity rangeof the assistive device 102 based on the received sensor data.

At 414, a relative position and a height of each of the plurality ofobjects with respect to the position of the user 110 of the assistivedevice 102 may be determined. The second circuitry 210 may be configuredto determine the relative position of each of the plurality of objectswith respect to the position of the user 110 of the assistive device102. The relative position of each of the plurality of objects may bedetermined based on the sensor data received in real time or near-realtime from the plurality of sensors 104 worn by the user 110.

At 416, the first template map with at least positional information ofthe plurality of objects may be updated, based on the received sensordata of the 3D real-world area within the first proximity range of theassistive device 102. The second circuitry 210 may be configured toupdate the first template map in real time or near-real time based onthe sensor data of the 3D real-world area.

At 418, a 3D digital model of the 3D real-world area within the firstproximity range may be generated. The second circuitry 210 may beconfigured to generate the3D digital model of the 3D real-world area,based on the received sensor data.

At 420, a scaling factor may be determined in accordance with the firstproximity range selected on the assistive device to map the plurality ofobjects within the first proximity range to the plurality of hapticelements in a defined region of the haptic feedback interface 112. Thesecond circuitry 210 may be configured to compute the scaling factor inaccordance with the first proximity range and the area of the definedregion of the haptic feedback interface 112.

At 422, a mapping of a plurality of objects within the first proximityrange to the plurality of haptic elements 218 in the defined region ofthe haptic feedback interface 112, may be established. The secondcircuitry 210 may be configured to establish the mapping of theplurality of objects to the plurality of haptic elements 218 in thedefined region of the haptic feedback interface 112, based on thereceived sensor data. In accordance with an embodiment, the generated 3Ddigital model may be utilized for the mapping of the plurality ofobjects within the first proximity range to the plurality of hapticelements 218.

At 424, a touch-discernible feedback may be generated on the hapticfeedback interface 112 based on the established mapping. The hapticfeedback generator 222 may be configured to generate thetouch-discernible feedback on the haptic feedback interface 112 based onthe established mapping. The touch-discernible feedback may comprise aplurality of differential touch-discernible cues to discern a relativeposition of each of the plurality of objects with respect to a positionof the user 110 of the assistive device 102. The haptic feedbackgenerator 222 also generates the plurality of differentialtouch-discernible cues to discern different identified object-types ofthe plurality of objects present within the first proximity range of theassistive device 102.

At 426, an output of an audio feedback may be controlled in combinationwith the touch-discernible feedback to enable non-visually discerning ofthe 3D real-world area within the first proximity range of the assistivedevice 102 by the user 110. The second circuitry 210 may be configuredto control the output of the audio feedback via the one or moreaudio-output devices 224 as the user 110 moves from a first location toa second location in the 3D real-world area within the first proximityrange.

At 428, the mapping of the plurality of objects to the plurality ofhaptic elements in the defined region of the haptic feedback interfacemay be updated. The haptic feedback generator 222 may be configured toupdate the mapping of the plurality of objects to the plurality ofhaptic elements 218 in the defined region of the haptic feedbackinterface 112. The update may be done based on a change in position ofone or more movable objects of the plurality of objects including theuser 110. The second circuitry 210 may be configured to detect thechange in real time or near-real time from the received sensor data.

At 430, the touch-discernible feedback may be dynamically updated on thehaptic feedback interface 112 based on the update of the mapping. Thehaptic feedback generator 222 may be configured to dynamically updatethe touch-discernible feedback on the haptic feedback interface 112based on the update of the mapping, in conjunction with the secondcircuitry 210 and the haptic feedback controller 220. The updatedtouch-discernible feedback comprises a change in the relative positionsof one or more of the plurality of objects with respect to the positionof the user 110 of the assistive device 102. Control passes to end 432.

In accordance with an exemplary aspect of the disclosure, a system fornon-visually discerning a three-dimensional (3D) real-world areasurrounding a user, such as the user 110 is disclosed. The system mayinclude the assistive device 102 (FIG. 1), which may comprise the hapticfeedback interface 112 (FIG. 1) comprising the plurality of hapticelements 218 (FIG. 2). The assistive device 102 may further comprise thefirst circuitry 208, the second circuitry 210, and the haptic feedbackgenerator 222 (FIG. 2). The first circuitry 208 may be configured toreceive sensor data of a 3D real-world area within a first proximityrange of the assistive device 102 from the plurality of sensors 104 thatare communicatively coupled to the assistive device 102. The secondcircuitry 210 may be configured to establish a mapping of a plurality ofobjects within the first proximity range to the plurality of hapticelements 218 in a defined region (e.g. the defined region 322) of thehaptic feedback interface 112, based on the received sensor data. Thehaptic feedback generator 222 may be configured to generate atouch-discernible feedback (e.g. the touch-discernible feedback 320) onthe haptic feedback interface 112 based on the established mapping. Thetouch-discernible feedback may comprise a plurality of differentialtouch-discernible cues to discern a relative position of each of theplurality of objects with respect to a position of the user 110 of theassistive device 102.

The present disclosure may be realized in hardware, or a combination ofhardware and software. The present disclosure may be realized in acentralized fashion, in at least one computer system, or in adistributed fashion, where different elements may be spread acrossseveral interconnected computer systems or the special-purpose device. Acomputer system or other special-purpose apparatus adapted to carry outthe methods described herein may be suited. The present disclosure maybe realized in hardware that comprises a portion of an integratedcircuit that also performs other functions.

The present disclosure may also be embedded in a computer programproduct, which comprises all the features that enable the implementationof the methods described herein, and which, when loaded in aspecial-purpose machine or computer system, is able to carry out thesemethods. Computer program, in the present context, means any expression,in any language, code or notation, of a set of instructions intended tocause a system with an information processing capability to perform aparticular function either directly, or after either or both of thefollowing: a) conversion to another language, code or notation; b)reproduction in a different material form.

While the present disclosure has been described with reference tocertain embodiments, it will be understood by those skilled in the artthat various changes may be made and equivalents may be substitutedwithout deviation from the scope of the present disclosure. In addition,many modifications may be made to adapt a particular situation ormaterial to the teachings of the present disclosure without deviationfrom its scope. Therefore, it is intended that the present disclosurenot be limited to the particular embodiment disclosed, but that thepresent disclosure will include all embodiments falling within the scopeof the appended claims.

1. An assistive device, comprising: a haptic feedback interfacecomprising a plurality of haptic elements; a first circuitry configuredto receive sensor data of a three-dimensional (3D) real-world area froma plurality of different types of sensors that are communicativelycoupled to the assistive device, wherein the sensor data is associatedwith a first proximity range of the assistive device; a second circuitryconfigured to: determine a scaling factor based on a surface area of adefined region of the haptic feedback interface; and establish a mappingof a plurality of objects within the first proximity range to at leastone haptic element of the plurality of haptic elements in the definedregion of the haptic feedback interface, wherein the mapping isestablished based on the received sensor data and the scaling factor;and a haptic feedback generator configured to generate atouch-discernible feedback on the haptic feedback interface based on theestablished mapping, wherein the touch-discernible feedback comprises aplurality of differential touch-discernible cues to discern a 3Darrangement of each object of the plurality of objects with respect to aposition of a user of the assistive device.
 2. The assistive deviceaccording to claim 1, wherein the touch-discernible feedback correspondsto at least one of a differential pressure-based, a differentialtemperature-based, a differential electric pulse-based, a differentialraised shape pattern-based touch-discernible feedback, or a combinationof different touch-discernible feedbacks.
 3. The assistive deviceaccording to claim 1, wherein the second circuitry is further configuredto identify an object-type of each object of the plurality of objectswithin the first proximity range of the assistive device, wherein theobject-type of each object is identified based on the received sensordata.
 4. The assistive device according to claim 3, wherein the hapticfeedback generator is further configured to generate the plurality ofdifferential touch-discernible cues to discern object-type of eachobject of the plurality of objects within the first proximity range ofthe assistive device.
 5. The assistive device according to claim 4,wherein the second circuitry is further configured to determine at leastone of a relative position or a height of each object of the pluralityof objects with respect to the position of the user of the assistivedevice for the 3D arrangement, and wherein the at least one of therelative position or the height of each object is determined based onthe sensor data received in real time or near-real time from theplurality of different types of sensors.
 6. The assistive deviceaccording to claim 4, wherein the second circuitry is further configuredto generate a three-dimensional (3D) digital model of the 3D real-worldarea within the first proximity range, wherein the 3D digital model isgenerated based on the received sensor data, and wherein the generated3D digital model is utilized for the mapping of the plurality of objectswithin the first proximity range to the plurality of haptic elements. 7.The assistive device according to claim 4, wherein the second circuitryis further configured to: acquire a first template map of the 3Dreal-world area within the first proximity range of the assistive devicefrom a server, wherein the first template is acquired based on theposition of the user; and update the first template map with at leastpositional information of the plurality of objects based on the sensordata of the 3D real-world area within the first proximity range of theassistive device, wherein the at least positional information of theplurality of objects is received from the plurality of different typesof sensors in real time or near-real time.
 8. The assistive deviceaccording to claim 1, wherein the second circuitry is further configuredto determine the scaling factor based on the first proximity rangeselected on the assistive device.
 9. The assistive device according toclaim 1, wherein the second circuitry is further configured to updatethe mapping of the plurality of objects within the first proximity rangeto the plurality of haptic elements in the defined region of the hapticfeedback interface, wherein the mapping is updated based on a change inposition of at least one movable object of the plurality of objects orthe user, and wherein the change in the position is detected in realtime or near-real time from the received sensor data.
 10. The assistivedevice according to claim 9, wherein the haptic feedback generator isfurther configured to dynamically update the touch-discernible feedbackon the haptic feedback interface based on the update of the mapping, andwherein the updated touch-discernible feedback comprises a change in therelative position of at least one object of the plurality of objectswith respect to the position of the user of the assistive device. 11.The assistive device according to claim 1, wherein the haptic feedbackgenerator is further configured to output an audio feedback incombination with the touch-discernible feedback to enable non-visualdiscernment of the 3D real-world area within the first proximity rangeof the assistive device by the user as the user moves from a firstlocation to a second location in the 3D real-world area within the firstproximity range.
 12. The assistive device according to claim 1, furthercomprises a learning engine in a memory of the assistive device, whereinthe second circuitry is further configured to determine at least onepattern in a plurality of user interactions on the haptic feedbackinterface over a period of time, and wherein the at least one pattern isdetermined based on a tracking of a usage pattern of the assistivedevice by the learning engine.
 13. The assistive device according toclaim 12, wherein the second circuitry is further configured to adaptthe mapping of the plurality of objects within the first proximity rangeto the plurality of haptic elements in the defined region of the hapticfeedback interface based on the determined at least one pattern.
 14. Theassistive device according to claim 1, wherein the haptic feedbackinterface is a haptic input/output interface.
 15. The assistive deviceaccording to claim 14 further comprising a haptic feedback controller,wherein the haptic feedback controller is configured to detect a hapticinput on the haptic feedback interface, and wherein the haptic input isdetected based on a press on at least one of the plurality ofdifferential touch-discernible cues generated in the touch-discerniblehaptic feedback.
 16. A method to provide assistance to a user,comprising: in an assistive device that comprises a first circuitry, asecond circuitry, a haptic feedback generator, and a haptic feedbackinterface that includes a plurality of haptic elements: receiving, bythe first circuitry, sensor data of a three-dimensional (3D) real-worldarea from a plurality of different types of sensors that arecommunicatively coupled to the assistive device, wherein the sensor datais associated with a first proximity range of the assistive device;determining a scaling factor based on a surface area of a defined regionof the haptic feedback interface; establishing, by the second circuitry,a mapping of a plurality of objects within the first proximity range toat least one haptic element of the plurality of haptic elements in thedefined region of the haptic feedback interface, wherein the mapping isestablished based on the received sensor data and the scaling factor;and generating, by the haptic feedback generator, a touch-discerniblefeedback on the haptic feedback interface based on the establishedmapping, wherein the touch-discernible feedback comprises a plurality ofdifferential touch-discernible cues to discern a 3D arrangement of eachobjects of the plurality of objects with respect to a position of a userof the assistive device, and wherein the touch-discernible feedback isfor non-visual discernment of the 3D real-world area surrounding theuser.
 17. The method according to claim 16, further comprisingidentifying, by the second circuitry, an object-type of each object ofthe plurality of objects within the first proximity range of theassistive device, wherein the object-type of each object is identifiedbased on the received sensor data.
 18. The method according to claim 17,further comprising generating, by the haptic feedback generator, theplurality of differential touch-discernible cues to discern theobject-type of each object of the plurality of objects within the firstproximity range of the assistive device.
 19. The method according toclaim 16, further comprising determining, by the second circuitry, atleast one of a relative position or a height of each object of theplurality of objects with respect to the position of the user of theassistive device for the 3D arrangement, wherein the at least one of therelative position or the height of each object is determined based onthe sensor data received in real time or near-real time from theplurality of different types of sensors worn by the user.
 20. The methodaccording to claim 16, further comprising generating, by the secondcircuitry, a three-dimensional (3D) digital model of the 3D real-worldarea within the first proximity range, wherein the 3D digital model isgenerated based on the received sensor data, and wherein the generated3D digital model is utilized for the mapping of the plurality of objectswithin the first proximity range to the plurality of haptic elements.21. The assistive device according to claim 1, wherein the scalingfactor indicates a relative adjustment of at least one of the size,shape, height, or position of the plurality of objects for a mapping ofthe plurality of objects to the plurality of haptic elements in thedefined region of the haptic feedback interface.